A Data-driven Approach to Predict Hand Positions for Two-hand Grasps of Industrial Objects

The wide spread use of 3D acquisition devices with highperformance processing tools has facilitated rapid generation of digital twin models for large production plants and factories for optimizing work cell layouts and improving human operator effectiveness, safety and ergonomics. Although recent advances in digital simulation tools have enabled users to analyze the workspace using virtual human and environment models, these tools are still highly dependent on user input to configure the simulation environment such as how humans are picking and moving different objects during manufacturing. As a step towards, alleviating user involvement in such analysis, we introduce a datadriven approach for estimating natural grasp point locations on objects that human interact with in industrial applications. Proposed system takes a CAD model as input and outputs a list of candidate natural grasping point locations. We start with generation of a crowdsourced grasping database that consists of CAD models and corresponding grasping point locations that are labeled as natural or not. Next, we employ a Bayesian network classifier to learn a mapping between object geometry and natural grasping locations using a set of geometrical features. Then, for a novel object, we create a list of candidate grasping positions and select a subset of these possible locations as natural grasping contacts using our machine learning model. We evaluate the advantages and limitations of our method by investigating the ergonomics of resulting grasp postures. INTRODUCTION The ever rising demand for innovative products, more sustainable production, and increasingly competitive global markets require constant adaptation and improvement of manufacturing strategies. Launching faster, obtaining higher return on investment, and delivering quality products, especially in demanding economic times and considering regulatory factors necessitates optimal planning and usage of manufacturing production capacity. Digital simulation of production plants and factories are invaluable tools for this purpose. Commercial software systems such as Siemens PLM Software Tecnomatix provide powerful 1 Copyright c © 2016 by ASME simulation functionality, and tools for visualizing and analyzing results of the simulations. Key aspects of optimizing manufacturing facilities that involve human operators include optimizing work cell layouts and activities for improving human operator effectiveness, safety and ergonomics. Examples of operations that are typically configured and analyzed in a simulation include humans picking and moving objects from one place to another, assembling a product consisting of multiple components in a factory, and using hand tools to perform maintenance tasks. One of the challenges in configuring such a simulation is in specifying the locations of the grasp points on objects that human interact with. The current approach relies on a manual process through which a user must specify the places where the human model should grasp each object. This is a tedious and time consuming process, and therefore a bottleneck in configuring large scale simulations. Therefore automated techniques for estimating natural grasp points are desirable. This paper presents a data driven approach for estimating natural looking grasp point locations on objects that human operators typically interact with in production facilities. These objects include mechanical tools, parts and components specific to products being manufactured or maintained such as automotive parts, etc. The proposed system takes a CAD model of an object as input, and outputs a list of candidate natural grasping point pairs. Each point pair consists of one point to place the palm of the left hand, and the other to place the palm of the right hand. We start with a crowdsourced database that consists of CAD models and corresponding grasping point locations that are labeled as natural or not. A Bayesian network classifier is used to learn a mapping between object geometry and natural grasping locations using a set of geometrical features. Then, for a new object not present in the database, we create a list of candidate grasping positions and select a subset of these possible locations as natural grasping contacts using the machine learning model. We evaluate the advantages and limitations of our method by investigating the ergonomics of resulting grasp postures. Our main contributions are as follows: 1. A novel grasp point estimation algorithm for two-hand grasps of objects that is tailored towards the digital simulation for production planning. 2. A set of geometric features to capture the natural appearance of two-hand grasps. 3. Ergonomics based evaluation of the performance. Section 2 presents related work. Section 3 presents an overview with technical details in Section 4. Results and discussion of the proposed approach are presented in Section 5, and conclusions are summarized in Section 6. RELATED WORK Analysis of holding objects has been explored in robotic grasping, computer graphics and fixture design. In robotics, researchers try to estimate contact points on an object based on sensor data that they have in the current setup. In general, problem is shrunk into hand grasping for certain types of robotic hands. Several examples of approaches adopted in the robotics fields can be found in [1–4]. In general, disadvantages of these methods are that they are all hardware dependent and main focus is daily used objects (such as bottles, cans and pans). On the other hand, researchers in computer graphics are interested in human grasping for simulation and animation purposes where natural looking grasping motions are desired. Some examples of grasping applications developed in computer graphics include [5, 6]. In fixture design, the grasping is studied to solve the problem of maintaining a specified position and orientation of an object in the presence of external disturbances (such as cutting forces in manufacturing). Since precision of the manufacturing process depends on workpiece stability, constraining the workpiece is critical [7]. Due to strict performance requirements, most approaches use physics based methods. Examples include [8] where Wang et al. uses force closure solutions for precision fixture design and a fixture layout design method based on largest simplex calculation [9]. While robotic grasping and computer graphics applications mainly focus on handling of everyday objects in our daily lives, fixture design examines holding mechanical objects firmly in place during manufacturing processes. In this paper, we focus on mechanical objects that will be handled in a factory environment as in fixture design. However, our main motivation is similar to computer graphics applications where achieving natural looking grasps is the main priority rather than firm and strict grasps. In these three areas, many techniques have been developed for grasping. The most common approaches to grasping problem include physics based analytical methods and data-driven methods. In literature, data-driven methods are presented to capture human decision making in grasping process. The examples include primitive based approaches [10, 11] where objects in database are represented as combinations of simple primitive shapes (such as cylinder, elipsoid, and cuboid) and shape matching methods [12] where suitable grasping pose is matched with the object to be manipulated. Other data driven methods are based on collecting human grasping data through labeling [2, 10, 12–14] or motion capture [15]. In physics based methods, the main idea is to find a set of feasible contact points that is optimum in terms of a pre-defined quality measure. Examples include [16] where authors present a method to select grasp points that minimize the contact forces. Similarly, Chinellato et al. [17] use geometrical constraints based on grasp polygon to evaluate the feasible point sets. A review of grasp quality measures can be found in [18]. While having an optimum solution for a certain physical ob2 Copyright c © 2016 by ASME FIGURE 1. Overview of the proposed methodology for data-driven grasping point estimation. jective could be suitable for robotic grasping applications and fixture design, we are, here, mainly interested in finding grasping configurations that are closest to natural human behavior. For this reason, instead of choosing a physical approach we use a data-driven method with crowdsourced labeling of human grasp.